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Abstract 


We examined the validity of using Compass® test scores and high school grade point 
average (GPA) for placing students in first-year college courses and for identifying students at 
risk of not succeeding. Consistent with other research, the combination of high school GPA and 
Compass scores performed better than either measure used alone. Results also indicate that, 
relative to Compass scores, the predictive strength of high school GPA decays with student age. 
We recommend using multiple measures for making course placement decisions and identifying 
students for intervention. 
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Validity Evidence for ACT Compass® Placement Tests 
Introduction 

ACT Compass assessments measure students’ skills in reading, writing, and mathematics. 
They are designed to assist institutions in placing college students in appropriate credit-bearing 
courses or in developmental or preparation courses. Compass scores can also be used to identify 
students who are enrolled in credit-bearing courses, but who may need additional academic 
support, such as tutoring or supplemental instruction. The Compass placement assessments are 
listed in Table 1, along with the number of assessments given at two- and four-year institutions 
in the 2011-2012 year (ACT, 2012). 

Table 1 

Number of Compass Assessments Administered at Two- and Four-Year Institutions, 2011-2012 


Compass test 

N 

Writing Skills 

1,350,835 

Reading 

1,456,301 

e-Write (writing essay) 

185,439 

Numerical Skills/Pre- algebra 

1,191,390 

Algebra 

1,212,731 

College Algebra 

142,892 

Trigonometry 

72,585 

Geometry 

25,099 


In this study, we evaluate the validity of using Compass placement test scores for making 
placement decisions and identifying students for additional academic support. We present three 
types of evidence: 

1 . Statistical measures of the overall strength of the relationship between the predictors 
- Compass scores and high school grade point average (HSGPA) - and success in 
standard first-year courses. This evidence is aligned to predicting success in standard 
first-year college-level courses. 
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2. Accuracy rates that estimate the proportion of students who are accurately placed into 
standard courses (likely to succeed in the standard courses) or developmental courses 
(unlikely to succeed in the standard courses). This evidence is related to placing 
students in developmental or college-level courses. 

3. Intervention hit rates that measure the accuracy of identifying the students least likely 
to succeed in standard courses. This evidence is related to identifying students in 
college-level courses who are at risk of not succeeding. 

We present this evidence for the five most frequently used Compass assessments - Writing 
Skills, Reading, Numerical Skills/Pre-Algebra, Algebra, and College Algebra. 

Previous studies have reported that Compass placement test scores and HSGPA, when 
used together, provide greater accuracy than either measure used alone (Noble, Schiel, & 
Sawyer, 2004; Scott-Clayton, 2012). This implies that institutions should use multiple measures 
(e.g., HSGPA and Compass scores), when possible. To accommodate the use of multiple 
measures for course placement, the Compass system offers a feature for weighting HSGPA and 
other measures, along with Compass scores, to create an overall placement score (ACT, 2006a). 
Among the institutions that use Compass, however, many students do not have high school grade 
information, either because they do not have a high school diploma 1 or because an official high 
school transcript is not readily available. Thus, single-predictor models are also used in practice 
and warrant research. This study considers three placement models: 

1 . Compass subject area score only, 

2. HSGPA only, and 


1 Among Compass examinees tested between September 1, 2011 and August 31, 2012 who reported their type of 
high school certificate, 62% reported having a high school diploma, 20% reported still being in high school, 13% 
reported having a GED, 4% reported not graduating from high school, and 2% reported having a certificate of 
completion or foreign secondary certificate (ACT, 2012). 
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3. Compass subject area score and HSGPA. 

As noted above, many incoming students do not have high school grade information. 
This lack of information is more common among adults who are entering or re-entering college 
several years after high school. Among Compass tests administered in 2011-2012 at two-year 
colleges, 31% of the examinees were 20 to 29 years old, and another 21% were age 30 or older 
(ACT, 2012). Because the predictive strength of the Compass placement tests and HSGPA 
might vary by student age, we compared results for traditional students (age 19 and under) and 
nontraditional students (age 20 and older). 

Data and Methods 

The study used course grade data collected from postsecondary institutions (two- and four-year 
colleges) on Compass examinees. The course grade data were obtained through ACT’s Course 
Placement Service, ACT’s Prediction Research Service, and other research partnerships with 
postsecondary institutions. ~ Several years of Compass and course grade data are included in the 
study. 3 

When submitting course grade data to ACT, institutions provide a course content code 
that indicates the course type (e.g., English Composition 1, Elementary Algebra, or Psychology). 
Institutions also indicate whether the course is standard, developmental (not credit-bearing), or 
honors. We excluded from the analysis courses that institutions classified as developmental or 
honors. Note that some institutions consider Arithmetic Skills, Elementary Algebra, and 
Intermediate Algebra to be developmental courses, but other institutions consider them to be 


For more information on ACT’s Course Placement Service, please see 
http://www.act.org/research/services/crsplace/. ACT’s Prediction Service is based on students who took the ACT 
Assessment; some of these students also took Compass. For more information on ACT’s Prediction Service, please 
see http://www.act.org/research/services/predict/, (ACT, 2006b). 

3 Individuals included in the study took Compass between 1995 and 2009. 
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standard courses. We included these three lower-level mathematics courses in the analyses when 
the institutions identified them as standard courses. 

We paired each course type with a Compass placement test commonly used for 
placement into the course. For each standard course, students were included in the analysis if 
they had grade data and a Compass score in the relevant subject. Table 2 contains the overall 
sample sizes for each course, as well as the Compass placement test that was used for each 
course. Institutions with fewer than 30 students were excluded from the analysis for each 
placement model. 4 This minimum number of student records ensured that the statistical models 
converged. 


4 Because of the minimum sample size requirement, the institutions used for model 2 (HSGPA only) and model 3 
(Compass & HSGPA) are a subset of the institutions used for model 1 (Compass only). Therefore, some of the 
differences across models may be due to sample differences. 
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Table 2 


Sample Size by Course Type and Predictor Set 


Course type 

Predictor set 
(see Note 1) 

Overall 

N Students 
Trad. 

Non-trad. 

ZVPlac. 
group 
(see Note 2) 

N 

Inst. 

English 

WS 

100,439 

53,719 

24,405 

2,492,578 

208 

Composition 1 

WS & HSGPA 

40,182 

29,848 

10,448 

771,598 

Speech/ 

WS 

7,040 

3,719 

2,384 

496,359 

32 

Rhetoric 

WS & HSGPA 

4,183 

2,937 

1,176 

194,695 

American 

R 

7,560 

4,096 

2,912 

607,893 

38 

History 

R & HSGPA 

3,426 

2,091 

1,311 

221,956 

Other History 

R 

R & HSGPA 

4,820 

2,312 

2,656 

1,705 

1,108 

580 

324,956 

102,651 

22 

Psychology 

R 

R & HSGPA 

29,855 

16,640 

18,030 

12,106 

9,627 

4,393 

1,457,615 

459,977 

101 

Sociology 

R 

R & HSGPA 

9,258 

5,240 

5,125 

3,474 

3,119 

1,678 

689,489 

280,052 

35 

Biology 

R 

R & HSGPA 

5,937 

3,573 

4,001 

2,793 

1,303 

744 

802,400 

243,500 

56 

Arithmetic 

PA 

3,413 

1,420 

1,642 

206,016 

13 

Skills 

PA & HSGPA 

1,184 

632 

547 

46,456 

Elementary 

PA 

12,965 

6,694 

2,667 

642,579 

49 

Algebra 

PA & HSGPA 

6,290 

4,495 

1,390 

266,143 

Intermediate 

AL 

10,452 

5,898 

2,358 

611,478 

44 

Algebra 

AL & HSGPA 

4,834 

3,699 

1,074 

206,576 

College 

AL 

15,843 

8,869 

2,943 

1,212,720 

92 

Algebra 

AL & HSGPA 

6,619 

4,751 

1,537 

309,463 

College 

CA 

6,261 

3,689 

619 

190,559 

44 

Algebra 

CA & HSGPA 

2,451 

2,005 

315 

65,276 


Notes. 


1. WS = Writing Skills; R = Reading; PA = Pre-Algebra; AL = Algebra; CA = College Algebra; HSGPA = 
high school grade point average. 

2. The “Placement group” consisted of the total number of examinees who completed Compass at an 
institution, even if the examinee did not later enroll at the institution. 


For each course, we examined the Compass-only predictor model for the overall sample. 
We examined the HSGPA-only and the Compass & HSGPA models for the subsample of 
students who reported their HSGPA when they took either Compass or the ACT® Assessment. 

We also fit separate predictive models for traditional and nontraditional students. From 
these models, we derived subgroup-specific logistic regression coefficients, accuracy rates, and 


intervention hit rates. 
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From Table 2, we note that each course type, with the exception of Arithmetic Skills, 
included more traditional students than nontraditional students. In English Composition, for 
example, 69% of the students with known age were traditional. This distribution is not reflective 
of the entire Compass-tested population (for which 52% of students are nontraditional). We also 
note that nontraditional students were less likely to have HSGPA data available. For example, 
for Intennediate Algebra, 63% of the traditional students had HSGPA data available compared to 
only 46% for the nontraditional students. 

Defining Course Success 

We defined success as earning a grade of B or higher; students earning C, D, or F grades, 
or who withdrew from the course, were classified as unsuccessful. 5 This dichotomization causes 
some loss of information (e.g., “A” grades are not distinguished from “B”s), but it corresponds to 
how student achievement is generally viewed: Is a student perfonning well? Over the years we 
have found that the distributions of first-year course grades, conditional on predictors, are 
generally not bell-shaped. Course grades below “C” are not assigned with much frequency, and 
the B criterion better reflects the grading practices college faculty use to discriminate student 
performance. In contrast, passing course grades of “C” and “D” indicate that students are 
“getting by” rather than performing well. 

In addition to this distinction between getting by and doing well, this definition of 
success was chosen because students who earn first-year grades of B or higher, on average, are 
much more likely to complete a postsecondary degree (Radunzel & Noble, 2012). For example, 
among 4-year students who earn a first-year GPA of at least 3.00 (B or higher grades, on 
average), 64% earned a Bachelor’s degree within six years versus just 27% for 4-year students 

5 In rare cases, colleges used a pass/fail grading system for a credit-bearing course. In these cases, “pass” was 
treated as successful and “fail” was treated as unsuccessful. 
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whose first-year GPA is less than 3.00. Similarly, among 2-year students who earn a first-year 
GPA of at least 3.00, 51% earned an Associate’s or Bachelor’s degree within six years versus 
just 19% for 2-year students whose first-year GPA is less than 3. 00. 6 Another reason for using 
this definition of success is that research (Noble & Sawyer, 2013) has shown that students who 
do not earn a B or higher in developmental courses do not benefit from taking them. 

Table 3 contains the proportions of students who were successful, by course type. 
Consistent with research on ACT-tested students (Allen, 2013), the success rates are highest in 
English courses (English Composition I and Speech/Rhetoric) and lowest in Biology. Relative to 
ACT-tested students, however, Compass-tested students typically have lower success rates in 
English Composition I (49% vs. 59%), College Algebra (42% to 47% vs. 49%), social science 
courses (41% to 48% vs. 53%), and Biology (35% vs. 47%). These differences are likely caused 
by differences between ACT-tested students and Compass-tested students in their level of 


academic preparation. 

Table 3 

B or Higher Success Rates, by Course 


Course type 

Compass test 

Proportion 

successful 

Overall Median* 

English Composition 1 

Writing Skills 

0.49 

0.49 

Speech/ Rhetoric 

Writing Skills 

0.54 

0.54 

American History 

Reading 

0.42 

0.32 

Other History 

Reading 

0.41 

0.42 

Psychology 

Reading 

0.45 

0.42 

Sociology 

Reading 

0.48 

0.41 

Biology 

Reading 

0.35 

0.32 

Arithmetic Skills 

Pre- Algebra 

0.43 

0.47 

Elementary Algebra 

Pre-Algebra 

0.38 

0.36 

Intermediate Algebra 

Algebra 

0.43 

0.43 

College Algebra 

Algebra 

0.42 

0.42 

College Algebra 

College Algebra 

0.47 

0.49 


*This is the median success rate across institutions. 


6 The degree completion percentages are obtained from the data set described in Radunzel and Noble (2012). 
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Statistical Modeling and Validity Measures 

Because of the hierarchical nature of the data (students nested within institutions), we 
used hierarchical logistic regression models to relate the predictors (Compass scores and 
HSGPA) to the dichotomous success criterion. The hierarchical model allows the intercept and 
slope to vary across institutions, which is an expected consequence of the variation across 
colleges in course content and grading standards. We then used the results of the logistic 
regression models to calculate three validity measures for each course and each institution: the 
logistic R, the accuracy rate, and the intervention hit rate. 

The logistic R measures the overall predictive strength of the model. This measure is 
derived in a way that is analogous to multiple R for multiple linear regression 7 , but is appropriate 
for logistic regression models. Logistic R is defined as the standard deviation of the estimated 
logit function (Allen & Le, 2008). The higher the logistic R is, the stronger the relationship is 

between the predictors and success in the course. We also present logistic regression coefficients 

8 

for each predictor. 

The accuracy rate estimates the proportion of students who are correctly placed into 
either standard or developmental courses. The logistic regression model yields a predicted 
probability of success for each student. Students with a predicted probability of 0.50 or more are 
treated as placed in the standard course, and students with a predicted probability less than 0.50 
are treated as placed in the developmental course. The accuracy rate is estimated as the 
proportion of students who are placed either in the standard course and are predicted to be 
successful, or in the developmental course and are predicted to be unsuccessful in the standard 
course. 

7 Note that multiple R is the square root of /A the coefficient of determination. 

s In a hierarchical model, the coefficients vary across institutions. We present the estimated typical value of each 
coefficient across institutions. 
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Dichotomizing the success criterion and setting a cutoff score on the predictor creates 
four categories of students: those above the cutoff score that are successful (true positives), those 
above the cutoff score that are not successful (false positives), those below the cutoff score that 
would be unsuccessful (true negatives), and those below the cutoff score that would be 
successful (false negatives). In practice, the outcome for students below the cutoff score is not 
observed, but statistical models like logistic regression allows researchers to estimate the 
probability of success in the standard course for all tested students. Summing the proportions of 
correctly placed students (the true positives and the true negatives) yields the accuracy rate (AR). 
AR is maximized at the score point associated with a 0.50 probability of success; this is known 
as the optimal cutoff score. For more details on the theoretical basis of the accuracy rate and 
how it is computed, we refer you to previous studies (ACT, 2004; Sawyer, 1996). 

The intervention hit rate measures the accuracy of predicting which students are least 
likely to succeed in standard courses. We defined the intervention hit rate as the proportion of 
students who are unsuccessful (do not earn a B or higher), among those ranking in the bottom 
10% on the placement test scores. This measure is helpful for understanding how effective the 
predictors are at identifying students for intervention. Similar to logistic R and the accuracy rate, 
a higher intervention hit rate is desirable. However, while increasing the percentage of students 
flagged increases the intervention hit rate, it also increases the percentage of students incorrectly 
identified as being in need of intervention. Previous research (Casillas, Allen, Kuo, Pappas, 
Hanson, & Robbins, 2011) examined intervention hit rates when different percentages (5, 10, 25) 
of bottom ranked students were flagged for intervention. The percentage flagged for additional 
academic support depends on institutional capacity. As the intensity and expense of the 
additional support increases, a lower percentage flagged should be used because institutional 
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capacity for delivering the additional support would be lower. For this analysis, we decided that 
flagging the bottom 10% was reasonable and reflects the situation where an institution can 
provide additional academic support for 10% of the students enrolled in a credit-bearing course. 
This percentage also provides a reasonable tradeoff between obtaining a high intervention hit 
rate and misidentifying students as being in need of intervention. 

Each validity measure is based on predicted probabilities from a logistic regression 
model. The sample of students who took each standard first-year course is not the same as the 
entire group of students at each institution that could potentially be placed into the course. The 
larger group is known as the placement group and includes students who were placed in 
developmental courses. The validity statistics should be calculated with respect to the placement 
group, as this is the group of students for whom placement and intervention decisions are made. 
To approximate the placement group for each college and each course, we included predicted 
probabilities for all Compass examinees at each college in the study - not just the examinees 
with grade data from credit-bearing courses. Table 2 includes the placement group sample sizes. 

For each course, each validity measure potentially varies by institution. To summarize 
the results, we present the median of each validity measure across institutions. 

As discussed earlier, we completed separate analyses for the overall group of students, 
for traditional students, and for non- traditional students. The figures for the traditional and 
nontraditional students allow us to examine what would happen if institutions actually used age- 
specific placement models. This approach supplements the approach taken in other studies (e.g., 
Radunzel & Noble, 2013; Sanchez, 2013) in which total group models were used to examine 
whether subgroup performances were overestimated or underestimated when the total group 
figures were used. 
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Limitations 

There are limitations of this study that should be understood before interpreting the 
results. First, although the sample sizes of institutions and students are generally large, the 
samples were not randomly drawn from the population of students who might enroll in 
developmental courses. Moreover, the student samples generally included a greater share of 
traditional students relative to nontraditional students, which is not reflective of the Compass- 
tested population. 

Second, we included institutions that had 30 or more students for each model. The 
institutions used for model 2 (HSGPA only) and model 3 (Compass & HSGPA) are a subset of 
the institutions used for model 1 (Compass only). Therefore, some of the differences across 
models may be due to sample differences rather than to the predictive strength of the model. 
Also, the sample sizes for the subgroup analyses (traditional and nontraditional students) at some 
institutions were small. Restricting the analysis to institutions that had at least 30 students in 
each group with full data (traditional and nontraditional students with both Compass scores and 
HSGPA) would have resulted in the same set of institutions used for each subgroup and each 
model. Doing so would have meant ignoring much of our data, however. For example, we had 
208 institutions with 30 or more students for the Composition 1 course analyses for Compass 
Writing Skills scores (Table 2); however, among those 208 institutions, there were only 117 
institutions with at least 30 traditional students and at least 30 nontraditional students. Limiting 
the analyses to those 117 institutions would have meant the exclusion of data from 91 
institutions. 

Third, we only report median validity statistics (logistic R , accuracy rates, and 
intervention hit rates) across institutions, focusing on “typical” results. Readers should keep in 
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mind that results vary across institutions due to sampling variability as well as systematic 
differences in course content, grading practices, and student performance. The full distributions 
of validity statistics are available upon request. 

Fourth, we used student-reported HSGPA rather than actual high school grades. We 
would expect that HSGPA obtained from official high school transcripts would have greater 
predictive strength than a self-report measure. However, prior research has shown that the self- 
reported course work and grades collected on the ACT are generally accurate relative to the 
actual information provided on student transcripts (ACT, 2013; Sawyer, Laing, & Houston, 
1988). Thus, the loss of predictive strength is not likely to be substantial. Furthermore, research 
has found that most institutions use only placement test scores to make placement decisions 
(Fields & Parsad, 2012). Only 21% of post-secondary institutions use other criteria, to include 
high school grades, when placing students into mathematics courses. For reading courses, only 
13% of institutions used other criteria. 

A fifth limitation is that we only examined models using Compass scores and HSGPA. 
Additional measures could yield greater predictive strength. Course or subject-specific high 
school grades, as well as level of high school courses taken, could provide more information than 
overall HSGPA. Measures of psychosocial factors have also been shown to be incrementally 
predictive of college course grades in models that include HSGPA and placement test scores 
(Robbins, Allen, Casillas, Peterson, & Le, 2006). Measuring multiple domains of academic and 
behavioral readiness has potential to improve the alignment of intervention and student services 
to student need (Allen, Robbins, & Sawyer, 2010). 
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Results 

In the pages that follow, we present results for the overall predictive strength of the three 
placement models, the predictive strength separately for traditional and nontraditional students, 
the conditional probabilities of success at various Compass scores and levels of HSGPA, course 
placement accuracy rates, and intervention hit rates. Results are presented for each course. 

Median Logistic R 

Table 4 presents the median logistic R for the three placement models for eleven first- 
year courses for the overall group of students. In most cases, the median logistic R for HSGPA 
is higher than that for Compass scores. In all cases, the predictive strength is highest when 
Compass scores and HSGPA are used jointly. For example, in Psychology courses, the median 
logistic R for Compass Reading scores is 0.47 and for HSGPA it is 0.63, but together the median 
logistic R is 0.77. 

Table 4 


Overall Predictive Strength, Median Logistic R 


Course type 

Compass test 

Compass 

Predictor set 

Compass & 
HSGPA HSGPA 

English Composition 1 

Writing Skills 

0.31 

0.57 

0.62 

Speech/ Rhetoric 

Writing Skills 

0.36 

0.69 

0.75 

American History 

Reading 

0.40 

0.69 

0.80 

Other History 

Reading 

0.47 

0.67 

0.81 

Psychology 

Reading 

0.47 

0.63 

0.77 

Sociology 

Reading 

0.54 

0.60 

0.77 

Biology 

Reading 

0.57 

0.79 

0.92 

Arithmetic Skills 

Pre-Algebra 

0.57 

0.34 

0.66 

Elementary Algebra 

Pre-Algebra 

0.36 

0.65 

0.80 

Intermediate Algebra 

Algebra 

0.47 

0.66 

0.84 

College Algebra 

Algebra 

0.41 

0.76 

0.88 

College Algebra 

College Algebra 

0.51 

0.76 

0.94 
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The results in Table 4 indicate variability in the relative predictive strength of Compass 
scores and HSGPA across different types of college courses. There is also variability across 
institutions, but only the median results are given here. 

The greatest increase in predictive strength due to the inclusion of HSGPA is observed 
for Elementary Algebra, where the median logistic R increased from 0.36 (for the Compass only 
model) to 0.80 (for the Compass + HSGPA model) - an increase of 122%. The greatest increase 
in predictive strength due to the inclusion of Compass score is observed for Arithmetic Skills, 
where the median logistic R increased from 0.34 (for the HSGPA only model) to 0.66 (for the 
Compass + HSGPA model) - an increase of 94%. 

The predictive strength of the multiple measure model ranged from a low of 0.62 in 
English Composition to a high of 0.94 in College Algebra. Across the different course types, the 
median predictive strength was 0.47 for Compass, 0.67 for HSGPA, and 0.80 for the multiple 
measures model. 

Standardized Logistic Regression Coefficients 

Table 5 presents standardized logistic regression coefficients for Compass scores and 
HSGPA when they are used singularly and jointly. 9 Results are presented separately for the 
overall group, traditional students, and nontraditional students. Note that the coefficients for 
HSGPA are typically higher for the traditional students than they are for the nontraditional 
students. Furthermore, the coefficients for Compass scores are often higher than those for 
HSGPA for the nontraditional students, a reversal of the pattern seen among the traditional 
students. For example, for traditional students enrolled in American History courses, the 
coefficient for Compass Reading scores in the single predictor model is 0.33 and the coefficient 


9 The results in Table 5 are fixed effects in the hierarchical model; they represent the results at typical institutions. 
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for HSGPA in the same model is 0.99. In contrast, the coefficient for Compass Reading scores is 


0.47 and the coefficient for HSGPA is 0.43 for the nontraditional students. 
Table 5 

Standardized Logistic Regression Coefficients 


Course type 


Single-pred. models 

Two-pred. model 

(Compass test) 

Subgroup 

Compass 

HSGPA 

Compass 

HSGPA 

English 

Overall 

0.34 

0.64 

0.22 

0.61 

Composition 1 

Traditional 

0.36 

0.76 

0.25 

0.72 

(Writing Skills) 

Nontraditional 

0.33 

0.38 

0.21 

0.36 

Speech/ Rhetoric 

Overall 

0.36 

0.74 

0.30 

0.69 

(Writing Skills) 

Traditional 

0.38 

0.88 

0.29 

0.82 


Nontraditional 

0.34 

0.34 

0.30 

0.31 

American History 

Overall 

0.41 

0.80 

0.39 

0.76 

(Reading) 

Traditional 

0.33 

0.99 

0.29 

0.96 


Nontraditional 

0.47 

0.43 

0.48 

0.38 

Other History 

Overall 

0.53 

0.72 

0.51 

0.69 

(Reading) 

Traditional 

0.54 

0.92 

0.52 

0.89 


Nontraditional 

0.60 

0.38 

0.72 

0.35 

Psychology 

Overall 

0.49 

0.68 

0.44 

0.63 

(Reading) 

Traditional 

0.48 

0.82 

0.39 

0.77 


Nontraditional 

0.52 

0.36 

0.47 

0.32 

Sociology 

Overall 

0.60 

0.65 

0.51 

0.60 

(Reading) 

Traditional 

0.55 

0.81 

0.41 

0.75 


Nontraditional 

0.64 

0.40 

0.62 

0.34 

Biology 

Overall 

0.59 

0.88 

0.52 

0.81 

(Reading) 

Traditional 

0.64 

0.94 

0.50 

0.86 


Nontraditional 

0.70 

0.63 

0.83 

0.56 

Arithmetic Skills 

Overall 

0.60 

0.38 

0.54 

0.30 

(Pre -Algebra) 

Traditional 

0.66 

0.62 

0.67 

0.51 


Nontraditional 

0.67 

0.15 a 

0.43 

0.08 b 

Elementary 

Overall 

0.42 

0.68 

0.38 

0.64 

Algebra 

Traditional 

0.47 

0.81 

0.43 

0.78 

(Pre-Algebra) 

Nontraditional 

0.41 

0.51 

0.32 

0.47 

Intermediate 

Overall 

0.52 

0.71 

0.48 

0.64 

Algebra 

Traditional 

0.61 

0.82 

0.52 

0.76 

(Algebra) 

Nontraditional 

0.54 

0.36 

0.44 

0.25 

College Algebra 

Overall 

0.44 

0.89 

0.39 

0.81 

(Algebra) 

Traditional 

0.45 

0.95 

0.36 

0.88 


Nontraditional 

0.51 

0.65 

0.43 

0.59 

College Algebra 

Overall 

0.56 

0.78 

0.47 

0.74 

(College Algebra) 

Traditional 

0.61 

0.87 

0.50 

0.82 


Nontraditional 

0.42 

0.50 

0.26 c 

0.47 


Note. a p=.16; b p=.45; c p=,12 
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The hierarchical logistic regression model also provides estimates of the variation in the 
regression coefficients across institutions. The estimated variances (and their standard errors) 
are shown in Tables A1 and A2 in the appendix. In these tables it is clear that in most cases the 
standardized logistic regression coefficients varied across institutions. When looking at the 
results in Table 5, the standardized coefficient may be larger for one predictor than it is for the 
other predictor at the “typical” institution. However, the size of each coefficient varies by 
institution. For example, for the English Composition I overall two-predictor model, the typical 
size of the Compass Writing Skills coefficient is 0.22 with a standard deviation of 0.17 across 
institutions while the typical size of the HSGPA coefficient is 0.61 with a standard deviation of 
0.15 across institutions. 

The unstandardized coefficients and their standard errors are presented Table A3 in the 
appendix. 

Conditional Probabilities of Success 

Because Compass scores and HSGPA are jointly predictive of success, the probability of 
success varies across levels of Compass scores and HSGPA. Table 6 shows the conditional 
probabilities of success when using five levels of Compass scores and three levels of HSGPA for 
the overall group of students across institutions. For example, in English Composition 1, 
students with a Writing Skills score of 30 and HSGPA of 2.0 have only a 0.23 probability of 
earning a grade of B or higher, but students with the same Writing Skills score and a HSGPA of 
4.0 have a 0.65 probability of earning a grade of B or higher. Note that these are median figures 
calculated using data from multiple institutions. These figures vary across institutions. 
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Table 6 


Conditional Success Probabilities by Compass Score and HSGPA 


Course type 

Compass test 

HSGPA 

30 

40 

Compass score 
50 60 70 

80 

90 

English Composition 1 

Writing Skills 

2.00 

0.23 

0.24 

0.26 

0.28 

0.29 

0.31 

0.32 



3.00 

0.43 

0.45 

0.47 

0.49 

0.51 

0.53 

0.55 



4.00 

0.65 

0.67 

0.69 

0.70 

0.72 

0.74 

0.75 

Speech/ Rhetoric 

Writing Skills 

2.00 

0.26 

0.28 

0.30 

0.33 

0.35 

0.37 

0.40 



3.00 

0.49 

0.51 

0.54 

0.57 

0.59 

0.62 

0.64 



4.00 

0.72 

0.74 

0.76 

0.78 

0.80 

0.81 

0.83 

American History 

Reading 

2.00 

0.05 

0.07 

0.09 

0.11 

0.14 

0.17 

0.21 



3.00 

0.13 

0.16 

0.20 

0.24 

0.29 

0.34 

0.40 



4.00 

0.27 

0.32 

0.38 

0.44 

0.51 

0.57 

0.63 

Other History 

Reading 

2.00 

0.05 

0.06 

0.08 

0.10 

0.13 

0.17 

0.21 



3.00 

0.13 

0.16 

0.20 

0.25 

0.30 

0.36 

0.43 



4.00 

0.30 

0.36 

0.42 

0.49 

0.55 

0.62 

0.68 

Psychology 

Reading 

2.00 

0.07 

0.09 

0.11 

0.14 

0.18 

0.23 

0.28 



3.00 

0.16 

0.20 

0.24 

0.30 

0.36 

0.42 

0.49 



4.00 

0.32 

0.38 

0.45 

0.52 

0.59 

0.65 

0.71 

Sociology 

Reading 

2.00 

0.07 

0.09 

0.12 

0.15 

0.20 

0.25 

0.31 



3.00 

0.14 

0.18 

0.23 

0.29 

0.36 

0.43 

0.51 



4.00 

0.27 

0.33 

0.40 

0.48 

0.56 

0.63 

0.70 

Biology 

Reading 

2.00 

0.02 

0.03 

0.04 

0.06 

0.08 

0.10 

0.14 



3.00 

0.07 

0.09 

0.12 

0.16 

0.22 

0.28 

0.35 



4.00 

0.19 

0.25 

0.32 

0.40 

0.48 

0.56 

0.65 

Arithmetic Skills 

Pre-Algebra 

2.00 

0.33 

0.39 

0.45 

0.52 

0.58 

0.64 

0.70 



3.00 

0.42 

0.49 

0.55 

0.61 

0.67 

0.72 

0.77 



4.00 

0.52 

0.59 

0.65 

0.70 

0.75 

0.80 

0.83 

Elementary Algebra 

Pre-Algebra 

2.00 

0.15 

0.17 

0.20 

0.23 

0.27 

0.30 

0.34 



3.00 

0.32 

0.36 

0.40 

0.45 

0.49 

0.54 

0.58 



4.00 

0.56 

0.60 

0.64 

0.69 

0.72 

0.76 

0.79 

Intermediate Algebra 

Algebra 

2.00 

0.16 

0.20 

0.24 

0.28 

0.33 

0.39 

0.45 



3.00 

0.34 

0.40 

0.46 

0.52 

0.58 

0.64 

0.69 



4.00 

0.59 

0.64 

0.70 

0.75 

0.79 

0.83 

0.86 

College Algebra 

Algebra 

2.00 

0.10 

0.12 

0.14 

0.17 

0.20 

0.23 

0.27 



3.00 

0.27 

0.31 

0.35 

0.40 

0.45 

0.50 

0.55 



4.00 

0.55 

0.60 

0.64 

0.69 

0.73 

0.77 

0.80 

College Algebra 

College Algebra 

2.00 

0.12 

0.15 

0.19 

0.23 

0.27 

0.32 

0.38 



3.00 

0.34 

0.40 

0.46 

0.52 

0.58 

0.64 

0.69 



4.00 

0.65 

0.71 

0.76 

0.80 

0.84 

0.87 

0.89 
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Median Accuracy Rates 

Table 7, on page 20, shows the median accuracy rates for the base rate and three predictor 
sets applied to the overall samples: 

1. No placement variables used and all students placed into the standard (non- 
developmental) course (this is the base rate), 

2. Compass score only, 

3. HSGPA only, and 

4. Compass and HSGPA used jointly. 

The general trend is that the accuracy rates for HSGPA are slightly higher than those for 
Compass scores, but using Compass scores and HSGPA together produces the highest accuracy 
rates. Using English Composition 1 as an example, the base rate indicates that the estimated 
probability of success in the course is 0.46 without using any placement variables. When using 
Compass Writing Skills scores, the accuracy rate increases to 0.61, 0.15 higher than the base 
rate, and when using HSGPA, the accuracy rate is 0.63, an increase of 0.17 over the base rate. 
When using Compass Writing Skills scores and HSGPA jointly, the accuracy rate increases to 
0.64, an increase of 0. 18 over the base rate. 

Note that the base rates in Table 7 are generally lower than the success rates reported in 
Table 3. This result occurs because the base rates are calculated using the placement group (see 
Table 2), but the success rates are calculated using only the students with course grade data. 
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Table 7 


Median Accuracy Rates, All Students 


Course type 

Compass test 

Base 

rate 

Compass 

Predictor set 

Compass 

HSGPA & HSGPA 

English Composition 

1 Writing Skills 

0.46 

0.61 

0.63 

0.64 

Speech/ Rhetoric 

Writing Skills 

0.53 

0.59 

0.65 

0.65 

American History 

Reading 

0.32 

0.68 

0.68 

0.70 

Other History 

Reading 

0.35 

0.66 

0.67 

0.70 

Psychology 

Reading 

0.40 

0.63 

0.66 

0.68 

Sociology 

Reading 

0.38 

0.65 

0.64 

0.67 

Biology 

Reading 

0.30 

0.70 

0.73 

0.74 

Arithmetic Skills 

Pre-Algebra 

0.46 

0.64 

0.58 

0.64 

Elementary Algebra 

Pre -Algebra 

0.35 

0.66 

0.67 

0.69 

Intermediate Algebra 

Algebra 

0.38 

0.65 

0.66 

0.69 

College Algebra 

Algebra 

0.35 

0.67 

0.70 

0.73 

College Algebra 

College Algebra 

0.53 

0.63 

0.66 

0.69 


Table 8 presents median accuracy rates, by predictor set, for the traditional students. 
Their accuracy rates tend to be slightly lower for Compass scores and higher for HSGPA than 
the results found in the overall analyses (Table 7), but the median accuracy rates for the joint use 
of Compass scores and HSGPA are quite similar to the overall analyses, with four higher, four 
lower, and four unchanged. 
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Table 8 


Median Accuracy Rates, Traditional Students 


Course type 

Compass 
placement test 

Base 

rate 

Compass 

Predictor set 
HSGPA 

Compass 
& HSGPA 

English Composition 1 

Writing Skills 

0.47 

0.60 

0.64 

0.64 

Speech/ Rhetoric 

Writing Skills 

0.51 

0.58 

0.65 

0.65 

American History 

Reading 

0.34 

0.66 

0.70 

0.71 

Other History 

Reading 

0.36 

0.64 

0.68 

0.69 

Psychology 

Reading 

0.39 

0.62 

0.66 

0.67 

Sociology 

Reading 

0.37 

0.65 

0.66 

0.68 

Biology 

Reading 

0.33 

0.67 

0.69 

0.70 

Arithmetic Skills 

Pre-Algebra 

0.37 

0.66 

0.63 

0.67 

Elementary Algebra 

Pre -Algebra 

0.34 

0.66 

0.69 

0.69 

Intermediate Algebra 

Algebra 

0.41 

0.63 

0.64 

0.66 

College Algebra 

Algebra 

0.41 

0.63 

0.67 

0.69 

College Algebra 

College Algebra 

0.49 

0.62 

0.67 

0.69 


Table 9 shows accuracy rates, by predictor set, for the nontraditional students. Jointly 
using Compass scores and HSGPA generally produced the highest median accuracy rates for the 
nontraditional students as well, but the results in Table 9 yield four interesting patterns. First, all 
twelve median base accuracy rates in the nontraditional student analyses are higher than those 
seen in the overall analyses (Table 7), and ten of the twelve are higher than those in the 
traditional student analyses (Table 8). What this means is that the older, nontraditional students 
have a higher estimated probability of success in these courses than do the younger, traditional 


students. 
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Table 9 


Median Accuracy Rates, Nontraditional Students 


Course type 

Compass 
placement test 

Base 

rate 

Compass 

Predictor set 

Compass 
HSGPA & HSGPA 

English Composition 1 

Writing Skills 

0.49 

0.58 

0.60 

0.60 

Speech/ Rhetoric 

Writing Skills 

0.55 

0.59 

0.57 

0.60 

American History 

Reading 

0.39 

0.63 

0.66 

0.67 

Other History 

Reading 

0.42 

0.61 

0.66 

0.66 

Psychology 

Reading 

0.44 

0.61 

0.61 

0.63 

Sociology 

Reading 

0.45 

0.63 

0.60 

0.65 

Biology 

Reading 

0.31 

0.69 

0.72 

0.72 

Arithmetic Skills 

Pre-Algebra 

0.52 

0.62 

0.57 

0.58 

Elementary Algebra 

Pre-Algebra 

0.41 

0.63 

0.62 

0.64 

Intermediate Algebra 

Algebra 

0.43 

0.63 

0.63 

0.66 

College Algebra 

Algebra 

0.40 

0.65 

0.66 

0.69 

College Algebra 

College Algebra 

0.56 

0.61 

0.62 

0.61 


Second, all but one of the median accuracy rates for the predictors used individually or 
jointly are lower than those in Tables 7 and 8. That is, Compass scores and HSGPA have lower 
predictive strength for nontraditional students than they do for traditional students. This suggests 
that other, unmeasured factors affect the performance of nontraditional more than they affect the 
performance of traditional students. Family and work commitments, among other factors, may 
explain this difference. 

Third, though the median accuracy rates for Compass scores for the non-traditional 
students are generally lower than those for the traditional students (Table 8), the differences 
between median accuracy rates for HSGPA for traditional and non-traditional students tend to be 
even larger. This suggests that the validity of HSGPA decays over time. This is related to the 
fourth point, which is that the median accuracy rates for Compass scores alone are equal to or 
higher than those for HSGPA alone for six of the twelve analyses. This occurs only once among 
the traditional students and three times in the overall analyses. This suggests that HSGPA 
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provides less information about the future perfonnance of nontraditional students than it does for 
traditional students. 

Median Intervention Hit Rates 

Table 10 presents the median intervention hit rates for the base rate and three predictor 

sets: 

1. No variables used to identify students for intervention (this is the base rate), 

2. Compass score only, 

3. HSGPA only, and 

4. Compass and HSGPA used jointly. 

The intervention hit rate is defined as the proportion of students expected to be 
unsuccessful in the course (i.e., earn a C or lower grade or withdraw from the course), among 
those scoring in the bottom 10% on the predictor variable(s) at each institution. In the context of 
the intervention hit rate, therefore, the base rate is the overall proportion of students expected not 
to succeed. Thus, the base rates for the intervention hit rates differ from the base rates for the 


accuracy rates given in Table 7. 
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Table 10 


Median Intervention Hit Rates, All Students 


Course type 

Compass test 

Base 

rate 

Compass 

Predictor set 

Compass 
HSGPA & HSGPA 

English Composition 1 

Writing Skills 

0.54 

0.69 

0.77 

0.78 

Speech/ Rhetoric 

Writing Skills 

0.47 

0.62 

0.75 

0.78 

American History 

Reading 

0.68 

0.82 

0.89 

0.90 

Other History 

Reading 

0.65 

0.86 

0.88 

0.92 

Psychology 

Reading 

0.60 

0.80 

0.84 

0.88 

Sociology 

Reading 

0.62 

0.84 

0.82 

0.87 

Biology 

Reading 

0.70 

0.89 

0.92 

0.94 

Arithmetic Skills 

Pre-Algebra 

0.54 

0.70 

0.67 

0.74 

Elementary Algebra 

Pre -Algebra 

0.65 

0.76 

0.87 

0.88 

Intermediate Algebra 

Algebra 

0.62 

0.70 

0.87 

0.88 

College Algebra 

Algebra 

0.65 

0.76 

0.91 

0.92 

College Algebra 

College Algebra 

0.47 

0.67 

0.82 

0.84 


The general pattern is that the intervention hit rates for HSGPA are slightly higher than 
those for Compass scores, but using Compass scores and HSGPA together produces the highest 
median intervention hit rates. Again using English Composition 1 as an example, the median 
intervention hit rate for Compass Writing Skills scores is 0.69, which is 0.15 higher than the base 
rate. The median hit rate for HSGPA is 0.77, and when using Compass Writing Skills scores and 
HSGPA jointly, the median hit rate increases to 0.78, an increase of 0.24 over the base rate. 

Table 1 1 presents the median intervention hit rates for the base rate and three predictor 
sets for traditional students. As with the median accuracy rates, the median intervention rates for 
the traditional students in Table 11 tend to parallel the results from the overall analyses (Table 
10), with the median intervention hits rates generally higher for HSGPA than they are for 
Compass scores, and with the highest rates associated with the joint use of Compass scores and 


HSGPA. 
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Table 1 1 


Median Intervention Hit Rates, Traditional Students 


Course type 

Compass 
placement test 

Base 

rate 

Compass 

Predictor set 

Compass 
HSGPA & HSGPA 

English Composition 

1 Writing Skills 

0.53 

0.66 

0.74 

0.75 

Speech/ Rhetoric 

Writing Skills 

0.49 

0.62 

0.77 

0.75 

American History 

Reading 

0.66 

0.78 

0.90 

0.90 

Other History 

Reading 

0.64 

0.81 

0.87 

0.88 

Psychology 

Reading 

0.61 

0.78 

0.85 

0.86 

Sociology 

Reading 

0.63 

0.83 

0.83 

0.86 

Biology 

Reading 

0.67 

0.85 

0.91 

0.92 

Arithmetic Skills 

Pre-Algebra 

0.63 

0.79 

0.77 

0.82 

Elementary Algebra 

Pre -Algebra 

0.66 

0.78 

0.87 

0.88 

Intermediate Algebra 

Algebra 

0.59 

0.73 

0.83 

0.84 

College Algebra 

Algebra 

0.59 

0.73 

0.87 

0.87 

College Algebra 

College Algebra 

0.49 

0.71 

0.82 

0.85 


Table 12 shows median intervention hit rates, by predictor set, for nontraditional 
students. The highest median intervention hit rates are generally those for the joint use of 
Compass scores and HSGPA, though there were two exceptions (Arithmetic Skills and College 
Algebra using Compass College Algebra). As with the median accuracy rates for nontraditional 
students, however, there are also interesting trends seen in the median intervention hit rate results 
for the nontraditional students. One is that the base rates are generally lower than those in the 
overall and traditional student analyses. Another is that although the median intervention hit 
rates are lower for both Compass scores and HSGPA when compared to the results for the 
traditional students, the magnitude of the differences tends to be larger for HSGPA, with seven 
of the rates for HSGPA being more than 0.10 lower for the non-traditional students than for the 
traditional students. Finally, the median intervention hit rates associated with the joint use of 
Compass scores and HSGPA were always lower than those seen in the overall analyses (Table 
10) and almost always lower than those for the traditional students (Table 1 1). 
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Table 12 


Median Intervention Hit Rates, Nontraditional Students 


Course type 

Compass test 

Base 

rate 

Compass 

Predictor set 
HSGPA 

Compass 
& HSGPA 

English Composition 1 

Writing Skills 

0.51 

0.63 

0.67 

0.67 

Speech/ Rhetoric 

Writing Skills 

0.45 

0.58 

0.63 

0.66 

American History 

Reading 

0.61 

0.74 

0.76 

0.80 

Other History 

Reading 

0.58 

0.77 

0.71 

0.83 

Psychology 

Reading 

0.56 

0.75 

0.72 

0.78 

Sociology 

Reading 

0.55 

0.78 

0.72 

0.83 

Biology 

Reading 

0.69 

0.88 

0.89 

0.93 

Arithmetic Skills 

Pre-Algebra 

0.48 

0.65 

0.52 

0.58 

Elementary Algebra 

Pre -Algebra 

0.59 

0.72 

0.74 

0.78 

Intermediate Algebra 

Algebra 

0.57 

0.70 

0.75 

0.78 

College Algebra 

Algebra 

0.60 

0.76 

0.86 

0.89 

College Algebra 

College Algebra 

0.44 

0.56 

0.79 

0.71 


Summary of Results 

Overall predictive strength 

■ When both Compass scores and HSGPA were available, the predictive strength of their 
joint use exceeded that of either measure used individually. 

■ For the overall group and traditional students (age 19 and under), HSGPA was typically a 
stronger predictor than Compass scores of earning a grade of B or higher. 

■ For the nontraditional students, Compass scores were stronger than HSGPA in predicting 
success in 7 of the 1 1 courses studied. 

■ The standardized logistic regression coefficients for HSGPA used alone or with Compass 
scores were higher for traditional students than they were for nontraditional students. 

■ The standardized logistic regression coefficients for Compass scores were similar for 


traditional and nontraditional students. 
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Accuracy rates 

■ When both Compass scores and HSGPA were available, the accuracy rate of their joint 
use exceeded that of either measure used individually in all but one course (Arithmetic 
Skills 10 ). 

■ For the overall group, the accuracy rates when using HSGPA alone tended to be higher 
than the accuracy rates when using Compass score alone. 

■ All of the differences between the median accuracy rates for Compass scores and the 
median accuracy rates for HSGPA were 0.06 or less, and for ten of the twelve courses the 
difference was 0.03 or less. 

■ Compared to the overall results, the differences between the median accuracy rates for 
Compass scores and the median accuracy rates for HSGPA were slightly larger for the 
traditional students and slightly smaller for the nontraditional students. 

Intervention hit rates 

■ When both Compass scores and HSGPA were available, intervention hit rates for 
Compass scores and HSGPA used together were higher than those for HSGPA alone or 
Compass scores alone in all but one course (Intermediate Algebra). 

■ For the overall group and traditional students, the hit rates when HSGPA was used alone 
were generally higher than the hit rates when using Compass scores alone. 

■ For nontraditional students, the differences between the median intervention hit rates for 
Compass scores and the median accuracy rates for HSGPA used alone were generally 
smaller than the differences seen with the traditional students. 


10 The decline in the accuracy rate from the Compass-only model to the joint model can be attributed to using a 
different sample of students to estimate the joint model. (The sample used for the joint model is a subset of the 
sample used for the Compass-only model). 
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Discussion 

We examined the strength of Compass placement tests and high school GPA for 
predicting success in first-year college courses. Predictive strength, course placement accuracy, 
and accuracy of identifying at-risk students are statistics that inform validity arguments for 
Compass and high school GPA. Consistent with other research, the combination of high school 
GPA and Compass scores performed better than either measure used alone. When HSGPA is 
available, its joint use with Compass scores helps institutions predict student success in courses, 
make placement decisions, and identify at-risk students. However, HSGPA is not always 
available, especially for nontraditional students enrolling at two-year institutions. 

The analyses for traditional and nontraditional students have provided new insights on the 
value of Compass scores for institutions serving nontraditional students (age 20 and older). The 
results indicate that the overall predictive strength of HSGPA tended to be lower for 
nontraditional students than it was for traditional students, and Compass scores were often the 
stronger predictor of academic performance for nontraditional students. The diminished 
predictive strength of HSGPA for nontraditional students was also seen in the analyses for 
accuracy rates and intervention hit rates. This may be due to the passage of time between high 
school completion and entry (or re-entry) into postsecondary education. The decline in the 
predictive strength of HSGPA makes Compass scores especially important for placement and 
intervention for nontraditional students. Because the study sample included fewer nontraditional 
students (relative to the overall Compass examinee population), the overall predictive strength of 
HSGPA is likely to be overestimated. On the other hand, the use of self-reported HSGPA versus 
official transcript HSGPA may have the effect of underestimating the predictive strength of 


HSGPA. 
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It is also important to note that many students in the Compass-tested population are still 
in high school. 11 Further research needs to address the question of whether high school student 
status moderates the relationship between HSGPA and college course success. We have 
observed that traditional/nontraditional status moderates the relationship of HSGPA and college 
course success, but further research could examine additional potential moderators of both 
Compass scores and HSGPA, including gender, full-time/part-time status, ethnicity, and first 
generation status. 

The study findings can also be helpful to Compass users who wish to use HSGPA in a 
multiple measures model and want to know how much weight to assign to each measure. The 
unstandardized regression coefficients in the appendix (Table A3) provide starting points for 
weighting Compass scores and HSGPA. For example, Figure 1 plots the probability of success 
in Intermediate Algebra, by Compass Algebra Score and HSGPA. 



Figure 1. Probability of Success in Intennediate Algebra, by Compass Algebra Score and 
HSGPA. 


1 1 Among Compass examinees who reported their type of high school certificate, 20% reported still being in high 
school (ACT, 2012). 
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Figure 1 is created by applying the logistic regression model to obtain the predicted 
probability of success ip), using the estimated regression coefficients in Table Al: 
exp(L) 

^ 1 + exp(L) 

L = —4.4027 + 0.0244 x Compass Algebra Score + 1.005 x HSGPA 

Note that the regression coefficients (-4.4027, 0.0244, and 1.005) are obtained from the 
appendix and are specific to Intermediate Algebra, without differentiating between traditional 
and nontraditional students. We recommend that individual institutions develop their own 
placement models from analyses of their own data. ~ The findings of this study, including the 
estimated regression coefficients, can be used as one consideration for determining optimal 
placement strategies. 

An alternative to formally assigning weights in multiple measures is to use “decision 
zones”: For students whose scores are within a band of the Compass cutoff score, HSGPA and 
other information can be used to make the placement decision. When other information (e.g., 
HSGPA or other test scores) is not readily available but can be obtained through additional data 
collection (e.g., requesting high school transcripts) or more testing, the decision zone approach 
offers some efficiency because the expense of obtaining additional information can be avoided 
for students who score outside of the decision zone. 

Despite the validity decay associated with HSGPA, models that use HSGPA and 
Compass jointly perform better than single-predictor models, even for nontraditional students. 
This is especially true for English Composition courses, where HSGPA was a stronger predictor 
of success for both traditional and nontraditional students. We therefore recommend using 

12 ACT’s Course Placement Service (http://www.act.org/research/services/crsplace/) provides a convenient 
mechanism for doing this. 
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multiple measures for making course placement decisions and identifying students for 
intervention whenever possible. 

Finally, recall that the HSGPA data used in this study were self-reported by students. 
Because of the stakes involved with course placement decisions, it is possible that self-reported 
HSGPA would experience a decline in predictive strength if routinely used as a placement 
measure. HSGPA data obtained from official student transcripts is therefore more trustworthy 
for use in making course placement decisions. Recent initiatives to streamline the collection and 
distribution of high school transcript information have potential to help colleges use HSGPA 
(and high school course work) for placement and intervention systematically and efficiently. 


13 Examples of state high school transcript initiatives include the Iowa Department of Education’s Electronic 
Transcript & Student Record Project and the Data Quality Campaign’s inclusion of “Student-level transcript 
information, including information on courses completed and grades earned” among the 10 essential elements of a 
comprehensive longitudinal data system (Laird, 2008). 



31 


References 

ACT. (2004). COMPASS Course Placement Sendee Interpretive Guide. Iowa City, IA: Author. 

ACT. (2006a). COMPASS/ESL reference manual. Iowa City, IA: Author. 

ACT. (2006b). Your college freshmen: An interpretive guide for the ACT Prediction Research 
Sendee. Iowa City, IA: Author. 

ACT. (2012). The ACT COMPASS PROGRAM Entering Student Descriptive Report, National 2 
and 4 Year Institutions, September 1, 2011 through August 31, 2012. Iowa City, IA: 
Author. 

ACT. (2013). Accuracy of Student-Reported High School GPA (Information Brief 2013-12). 
Iowa City, IA. 

Allen, J. (2013). Updating the ACT College Readiness Benchmarks. (ACT Research Report 
Series No. 2013-6). Iowa City, IA: ACT, Inc. 

Allen, J., & Le, H. (2008). An additional measure of overall effect size for logistic regression 
models. Journal of Educational and Behavioral Statistics, 33(4): 416-441. 

Allen, J., Robbins, S., & Sawyer, R. (2010). Can measuring psychosocial factors promote college 
success? Applied Measurement in Education, 23: 1-22. 

Casillas, A., Allen, J., Kuo, Y. L., Pappas, S., Hanson, M. A., & Robbins, S. (2011). 
Development and validation of ENGAGE grades 6-9. (ACT Research Report Series No. 
2011-1). Iowa City, IA: ACT, Inc. 

Fields, R., & Parsad, B. (2012). Tests and Cut Scores Used for Student Placement in 
Postsecondary Education: Fall 2011. Washington, DC: National Assessment Governing 
Board. 

Laird, E. (2008). Developing and Supporting P-20 Education Data Systems: Different States, 
Different Models. Downloaded October 2, 2013 from 

http://www.dataqualitycampaign.org/files/events/resources/developingissuebriefO 1 1 508.p 
df. 

Noble, J. P., & Sawyer, R. L. (2013). A study of the effectiveness of developmental courses for 
improving success in college. (ACT Research Report Series No. 2013-1). Iowa City, IA: 
ACT, Inc. 

Noble, J. P., Schiel, J. L., & Sawyer, R. L. (2004). Assessment and college course placement: 
Matching students with appropriate instruction. In J. E. Wall & G. R. Walz (Eds.), 
Measuring up: Assessment issues for teachers, counselors, and administrators (pp. 297- 
311). Greensboro, NC: ERIC Counseling & Student Services Clearinghouse and the 
National Board of Certified Counselors. 



32 


Radunzel, J., & Noble, J. (2012). Predicting long-term college success through degree 
completion using ACT Composite Score, ACT Benchmarks, and high school grade point 
average. (ACT Research Report Series No. 2012-5). Iowa City, IA: ACT, Inc. 

Radunzel, J., & Noble, J. (2013). Differential effects on student demographic groups ACT 
College Readiness Assessment Composite Score, ACT Benchmarks, and high school 
grade point average for predicting long-term college success through degree completion. 
(ACT Research Report Series No. 2013-5). Iowa City, IA: ACT, Inc. 

Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential 
effects of motivational and skills, social, and self-management measures from traditional 
predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. 

Sanchez, E. (2013). Differential effects of using ACT College Readiness Assessment Composite 
Score and high school GPA to predict first-year college GPA among racial/ethnic, 
gender, and income groups. (ACT Research Report Series No. 2013-4). Iowa City, IA: 
ACT, Inc. 

Sawyer, R. L. (1996). Decision theory models for validating course placement tests. Journal of 
Educational Measurement, 33, 271-290. 

Sawyer, R.L., Laing, J., & Houston, M. (1988). Accuracy of self-reported high school courses 
and grades of college-bound students (ACT Research Report No. 88-1). Iowa City, IA: 
American College Testing Program. 

Scott-Clayton, J. (2012). Do high-stakes placement exams predict college success? (CCRC 
Working Paper No. 41). New York, NY: Columbia University, Teachers College, 
Community College Research Center. 



33 


Appendix 

Tables Al, A2, and A3 



34 



35 


Table A1 


Estimated Variances across Institutions of Standardized Logistic Regression Coefficients in the 
Single-Predictor Models (Standard Errors in Parentheses) 


Course Type 


Compass 

HSGPA 

(Compass test) 

Subgroup 

Intercept 

Slope 

Intercept 

Slope 

English 
Composition 1 
(Writing Skills) 

Overall 

Traditional 

Nontraditional 

0.259 (0.030) 
0.366 (0.048) 
0.282 (0.040) 

0.039 (0.007) 
0.034 (0.008) 
0.031 (0.011) 

0.271 (0.036) 
0.301 (0.044) 
0.341 (0.055) 

0.027 (0.008) 
0.026 (0.010) 
0.013 (0.009) 

Speech/ 
Rhetoric 
(Writing Skills) 

Overall 

Traditional 

Nontraditional 

0.139 (0.049) 
0.152 (0.061) 
0.079 (0.055) 

0.047 (0.023) 
0.016(0.017) 
0.056 (0.050) 

0.267 (0.094) 
0.272 (0.103) 
0.165 (0.095) 

0.040 (0.023) 
0.025 (0.024) 

American 

History 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.266 (0.077) 
0.330 (0.109) 
0.378 (0.142) 

0.004 (0.010) 
0.006 (0.015) 

0.417(0.135) 
0.370 (0.136) 
0.530 (0.208) 

0.080 (0.049) 
0.124 (0.089) 

Other History 
(Reading) 

Overall 

Traditional 

Nontraditional 

0.227 (0.088) 
0.182 (0.077) 
0.487 (0.204) 

0.073 (0.043) 
0.042 (0.044) 

0.352 (0.142) 
0.206 (0.103) 
0.904 (0.398) 

0.000 (0.000) 
0.126 (0.111) 

Psychology 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.316 (0.053) 
0.341 (0.064) 
0.382 (0.077) 

0.057 (0.016) 
0.062 (0.020) 
0.048 (0.022) 

0.502 (0.091) 
0.475 (0.094) 
0.477 (0.112) 

0.060 (0.017) 
0.051 (0.020) 
0.009 (0.011) 

Sociology 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.278 (0.080) 
0.353 (0.112) 
0.344 (0.128) 

0.029 (0.018) 
0.033 (0.023) 
0.060 (0.046) 

0.348 (0.108) 
0.320 (0.104) 
0.575 (0.226) 

0.039 (0.024) 
0.057 (0.041) 
0.001 (0.014) 

Biology 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.145 (0.044) 
0.226 (0.072) 
0.166 (0.085) 

0.075 (0.036) 
0.069 (0.050) 
0.083 (0.074) 

0.215 (0.072) 
0.173 (0.073) 
0.406 (0.182) 

0.037 (0.030) 
0.067 (0.042) 
0.051 (0.089) 

Arithmetic 

Skills 

(Pre-Algebra) 

Overall 

Traditional 

Nontraditional 

0.344 (0.152) 
0.560 (0.282) 
0.222 (0.116) 

0.005 (0.014) 
0.010 (0.030) 
0.077 (0.086) 

0.391 (0.216) 
0.386 (0.242) 
0.378 (0.292) 

0.080 (0.088) 
0.097 (0.120) 

Elementary 

Algebra 

(Pre-Algebra) 

Overall 

Traditional 

Nontraditional 

0.259 (0.074) 
0.130 (0.049) 
0.492 (0.174) 

0.055 (0.021) 
0.036 (0.026) 
0.036 (0.025) 

0.314(0.100) 
0.196 (0.073) 
0.492 (0.197) 

0.026 (0.019) 
0.005 (0.024) 

Intermediate 

Algebra 

(Algebra) 

Overall 

Traditional 

Nontraditional 

0.257 (0.070) 
0.216 (0.072) 
0.578 (0.189) 

0.040 (0.016) 
0.026 (0.022) 
0.021 (0.026) 

0.459 (0.143) 
0.392 (0.135) 
0.732 (0.287) 

0.045 (0.030) 
0.022 (0.028) 
0.067 (0.064) 

College Algebra 
(Algebra) 

Overall 

Traditional 

Nontraditional 

0.443 (0.080) 
0.438 (0.089) 
0.636 (0.151) 

0.058 (0.016) 
0.045 (0.018) 
0.030 (0.021) 

0.490 (0.103) 
0.482 (0.113) 
0.771 (0.214) 

0.073 (0.033) 
0.079 (0.039) 
0.069 (0.069) 

College Algebra 

(College 

Algebra) 

Overall 

Traditional 

Nontraditional 

0.281 (0.073) 
0.307 (0.092) 
0.294 (0.142) 

0.088 (0.032) 
0.062 (0.037) 
0.045 (0.088) 

0.417(0.126) 
0.471 (0.149) 
0.320 (0.198) 

0.048 (0.042) 
0.046 (0.041) 
0.115 (0.144) 


Note. Blank spaces indicate that the model did not detect statistically significant variation across institutions. 
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Table A2 


Estimated Variances across Institutions of Standardized Logistic Regression Coefficients in the 
Two-Predictor Models (Standard Errors in Parentheses) 


Course type 
(Compass test) 

Subgroup 

Intercept 

Compass 

HSGPA 

English 
Composition 1 
(Writing Skills) 

Overall 

Traditional 

Nontraditional 

0.2759 (0.0363) 
0.3136 (0.0456) 
0.3318 (0.0544) 

0.0296 (0.0090) 
0.0207 (0.0085) 
0.0171 (0.0142) 

0.0237 (0.0075) 
0.0188 (0.0089) 
0.0124 (0.0091) 

Speech/ Rhetoric 
(Writing Skills) 

Overall 

Traditional 

Nontraditional 

0.2668 (0.0962) 
0.2915 (0.1085) 
0.1577 (0.0954) 

0.0387 (0.0351) 
0.0781 (0.0825) 

0.0321 (0.0198) 
0.0175 (0.0201) 

American History 
(Reading) 

Overall 

Traditional 

Nontraditional 

0.4338 (0.1394) 
0.3794 (0.1374) 
0.6031 (0.2438) 

0.1070 (0.1384) 

0.0725 (0.0478) 
0.1085 (0.0843) 

Other History 
(Reading) 

Overall 

Traditional 

Nontraditional 

0.3091 (0.1294) 
0.1987 (0.0986) 
0.9061 (0.3983) 

0.0764 (0.0725) 
0.0118 (0.0312) 

0.0748 (0.0878) 

Psychology 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.5348 (0.0965) 
0.5032 (0.0985) 
0.5416 (0.1258) 

0.0351 (0.0149) 
0.0530 (0.0225) 
0.0032 (0.0144) 

0.0532 (0.0164) 
0.0385 (0.0175) 
0.0094(0.0119) 

Sociology 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.3434 (0.1062) 
0.3183 (0.1024) 
0.5569 (0.2269) 

0.0151 (0.0194) 
0.0160 (0.0169) 
0.1153 (0.1113) 

0.0343 (0.0220) 
0.0451 (0.0371) 

Biology 

(Reading) 

Overall 

Traditional 

Nontraditional 

0.1864 (0.0666) 
0.1426 (0.0642) 
0.4684 (0.2065) 

0.0524 (0.0516) 

0.0231 (0.0271) 
0.0460 (0.0363) 
0.0398 (0.0855) 

Arithmetic Skills 
(Pre -Algebra) 

Overall 

Traditional 

Nontraditional 

0.3861 (0.2132) 
0.3863 (0.2421) 
0.4049 (0.3028) 

0.0179 (0.0366) 
0.0581 (0.1221) 
0.0247 (0.0801) 

0.0837 (0.0903) 
0.1123 (0.1293) 

Elementary 
Algebra 
(Pre -Algebra) 

Overall 

Traditional 

Nontraditional 

0.3052 (0.0968) 
0.2141 (0.0797) 
0.4357 (0.1828) 

0.0260 (0.0146) 
0.0057 (0.0095) 
0.0907 (0.0528) 

0.0191 (0.0166) 
0.0031 (0.0226) 

Intermediate 

Algebra 

(Algebra) 

Overall 

Traditional 

Nontraditional 

0.4871 (0.1519) 
0.4214(0.1445) 
0.7316 (0.2948) 

0.0423 (0.0266) 
0.0192 (.02320) 
0.0634 (0.0565) 

0.0472 (0.0320) 
0.0235 (0.0297) 
0.0464 (0.0609) 

College Algebra 
(Algebra) 

Overall 

Traditional 

Nontraditional 

0.5164 (0.1098) 
0.5381 (0.1271) 
0.7279 (0.2014) 

0.0407 (0.0173) 
0.0495 (0.0221) 
0.0096 (0.0210) 

0.0746 (0.0334) 
0.0835 (0.0412) 
0.0719(0.0670) 

College Algebra 
(College Algebra) 

Overall 

Traditional 

Nontraditional 

0.3960 (0.1219) 
0.4105 (0.1342) 
0.3083 (0.2000) 

0.0771 (0.0440) 
0.0834 (0.0507) 
0.0723 (0.1678) 

0.0512(0.0434) 
0.0438 (0.0407) 
0.0688 (0.1359) 


Note. Blank spaces indicate that the model did not detect statistically significant variation across institutions. 
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Table A3 


Estimated Unstandardized Logistic Regression Fixed-Effects Coefficients (Standard Errors in 
Parentheses) 


Course type 
(Compass test) 

Subgroup 

Intercept 

Compass 

HSGPA 

English 
Composition 1 
(Writing Skills) 

Overall 

Traditional 

Nontraditional 

-3.2785 (0.0813) 
-3.8958 (0.0963) 
-2.1722 (0.1296) 

0.0078 (0.0007) 
0.0084 (0.0007) 
0.0065 (0.0011) 

0.9200 (0.0223) 
1.1276 (0.0261) 
0.5456 (0.0353) 

Speech/ Rhetoric 
(Writing Skills) 

Overall 

Traditional 

Nontraditional 

-3.3527 (0.2249) 
-4.0510 (0.2609) 
-1.7268 (0.3294) 

0.0106 (0.0018) 
0.0099 (0.0017) 
0.0078 (0.0030) 

0.9952 (0.0682) 
1.2330 (0.0787) 
0.4760 (0.1005) 

American History 
(Reading) 

Overall 

Traditional 

Nontraditional 

-5.4835 (0.3438) 
-5.9795 (0.4383) 
-4.8264 (0.5726) 

0.0256 (0.0031) 
0.0194 (0.0039) 
0.0301 (0.0056) 

0.9282 (0.0676) 
1.2784 (0.0950) 
0.5767(0.1012) 

Other History 
(Reading) 

Overall 

Traditional 

Nontraditional 

-5.8559 (0.4353) 
-6.3426 (0.4922) 
-5.6976 (0.9653) 

0.0268 (0.0044) 
0.0248 (0.0047) 
0.0416 (0.0104) 

1.0497 (0.0832) 
1.3019(0.1034) 
0.5440 (0.1551) 

Psychology 

(Reading) 

Overall 

Traditional 

Nontraditional 

-5.2787 (0.1844) 
-5.6743 (0.1973) 
-4.0511 (0.2931) 

0.0275 (0.0016) 
0.0236 (0.0018) 
0.0295 (0.0029) 

0.9247 (0.0429) 
1.1453 (0.0426) 
0.4753 (0.0548) 

Sociology 

(Reading) 

Overall 

Traditional 

Nontraditional 

-5.2252 (0.2725) 
-5.4600 (0.3225) 
-4.6442 (0.5032) 

0.0311 (0.0024) 
0.0243 (0.0030) 
0.0357 (0.0052) 

0.8204 (0.0550) 
1.0728 (0.0704) 
0.5287 (0.0908) 

Biology 

(Reading) 

Overall 

Traditional 

Nontraditional 

-7.2823 (0.3716) 
-7.4851 (0.4137) 
-7.8534 (0.9414) 

0.0340 (0.0036) 
0.0326 (0.0039) 
0.0530 (0.0095) 

1.2064 (0.0760) 
1.3266 (0.0896) 
0.7973 (0.1547) 

Arithmetic Skills 
(Pre -Algebra) 

Overall 

Traditional 

Nontraditional 

-2.2506 (0.3775) 
-3.7439 (0.5399) 
-1.3220 (0.5108) 

0.0254 (0.0040) 
0.0312 (0.0058) 
0.0226 (0.0060) 

0.3946 (0.1119) 
0.7833 (0.1703) 
0.1272(0.1613) 

Elementary 
Algebra 
(Pre -Algebra) 

Overall 

Traditional 

Nontraditional 

-4.2724 (0.2004) 
-5.0575 (0.2370) 
-3.1182 (0.3471) 

0.0182 (0.0021) 
0.0213 (0.0020) 
0.0142 (0.0041) 

0.9903 (0.0531) 
1.1953 (0.0667) 
0.7174 (0.1020) 

Intermediate 

Algebra 

(Algebra) 

Overall 

Traditional 

Nontraditional 

-4.4027 (0.2419) 
-5.0561 (0.2795) 
-2.3824 (0.3839) 

0.0244 (0.0030) 
0.0270 (0.0029) 
0.0218 (0.0053) 

1.0053 (0.0640) 
1.1937(0.0764) 
0.3659 (0.1313) 

College Algebra 
(Algebra) 

Overall 

Traditional 

Nontraditional 

-5.1914(0.2155) 
-5.2143 (0.2523) 
-4.6022 (0.4111) 

0.0198 (0.0020) 
0.0181 (0.0023) 
0.0220 (0.0033) 

1.1985 (0.0584) 
1.2557 (0.0706) 
0.0987(0.1164) 

College Algebra 
(College Algebra) 

Overall 

Traditional 

Nontraditional 

-5.3309 (0.3614) 
-5.9113 (0.4200) 
-3.2469 (0.8042) 

0.0249 (0.0038) 
0.0258 (0.0041) 
0.0156 (0.0084) 

1.3034 (0.0984) 
1.4287 (0.1135) 
0.8545 (0.2385) 
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